RNN has many different variations with distinct structures. Shown in Fig. 9.10 is one of the most common architectures. On the left is the schematic of a unit. The one in the middle shows a more detailed flow of the data through the unit. The subplot on the right illustrates how the unit unfolds over time.

Figure 9.10: Structure and workflow of RNN

The symbols in Fig. 9.10 have the following meanings:
$x^t$$x^t$x^(t)x^{t}$x^t$ is the output at step $t$$t$tt$t$ in the time series;
$h^t$$h^t$h^(t)h^{t}$h^t$ is the state of the hidden layer at $t$$t$tt$t$, which is determined by both $x^t$$x^t$x^(t)x^{t}$x^t$ and $h^{t-1}$$h^{t-1}$h^(t-1)h^{t-1}$h^{t-1}$ according to the above network architecture;
$o^t$$o^t$o^(t)o^{t}$o^t$ is the output at $t$$t$tt$t$, which is determined by the hidden state at the current step $t$$t$tt$t$;
$L^t$$L^t$L^(t)L^{t}$L^t$ is the loss function at $t$$t$tt$t$;
$y^t$$y^t$y^(t)y^{t}$y^t$ is the actual output (or label) at $t$$t$tt$t$;
${\tilde{y}}^t$${\tilde{y}}^t$tilde(y)^(t)\tilde{y}^{t}${\tilde{y}}^t$ is the predicted output at $t$$t$tt$t$;
${\sigma }_1$${\sigma }_1$sigma_(1)\sigma_{1}${\sigma }_1$ and ${\sigma }_2$${\sigma }_2$sigma_(2)\sigma_{2}${\sigma }_2$ are the hidden layer activation function and output activation function, respectively;
$U,W$$U,W$U,WU, W$U,W$, and $V$$V$VV$V$ are the arrays that include the network weights to be predicted.

These weights, i.e., $U,W$$U,W$U,WU, W$U,W$, and $V$$V$VV$V$, are shared through time. That is, though there are "multiple units" as the RNN is unfolded over time, there is essentially one unit. Thus, the backpropagation of RNN updates the weights of the same unit.

The above symbols were given without discussing the shape of the input $x$$x$xx$x$, which is handled as one sample here. In many cases, $x$$x$xx$x$ has multiple attributes and thus needs to be formulated using a 1D array. In the following deductions for backpropagation, we will represent $x$$x$xx$x$ as a column array with $J$$J$JJ$J$ elements (for the $J$$J$JJ$J$ attributes). In this way, the derived equations can be easily used for coding. The output $y$$y$yy$y$ (actual) and $\tilde{y}$$\tilde{y}$tilde(y)\tilde{y}$\tilde{y}$ (predicted) are 1D arrays with a shape of $K\times 1$$K\times 1$K xx1K \times 1$K\times 1$. In implementation, we will still need to determine the shape of $\overrightarrow{h}$$\overrightarrow{h}$vec(h)\vec{h}$\overrightarrow{h}$. Let us assume $\overrightarrow{h}$$\overrightarrow{h}$vec(h)\vec{h}$\overrightarrow{h}$ has a shape of $N\times 1$$N\times 1$N xx1N \times 1$N\times 1$. The shapes of other weight arrays can be determined by those of $\overrightarrow{x},\overrightarrow{y}$$\overrightarrow{x},\overrightarrow{y}$vec(x), vec(y)\vec{x}, \vec{y}$\overrightarrow{x},\overrightarrow{y}$, and $\overrightarrow{h}$$\overrightarrow{h}$vec(h)\vec{h}$\overrightarrow{h}$. The data flow, especially the change of the shapes of arrays for data and weights, can be tracked in the following deductions.

Let us first work on the forward pass. The following three operations will turn the input ${\overrightarrow{x}}_{J\times 1}$${\overrightarrow{x}}_{J\times 1}$vec(x)_(J xx1)\vec{x}_{J \times 1}${\overrightarrow{x}}_{J\times 1}$ into $\tilde{y}$$\tilde{y}$tilde(y)\tilde{y}$\tilde{y}$.

where ${\sigma }_1$${\sigma }_1$sigma_(1)\sigma_{1}${\sigma }_1$ and ${\sigma }_2$${\sigma }_2$sigma_(2)\sigma_{2}${\sigma }_2$ are the two activation functions used for the hidden layer and output layer, respectively.
The loss function at time step ${\ell }^t$${\ell }^t$ℓ^(t)\ell^{t}${\ell }^t$ measures the distance between ${\tilde{\overrightarrow{y}}}^t$${\tilde{\overrightarrow{y}}}^t$tilde(vec(y))^(t)\tilde{\vec{y}}^{t}${\tilde{\overrightarrow{y}}}^t$ and ${\overrightarrow{y}}^t$${\overrightarrow{y}}^t$vec(y)^(t)\vec{y}^{t}${\overrightarrow{y}}^t$. For example, the log likelihood function is usually used for classification tasks, and the following mean square root error function is used for regression tasks.

RNN's backward pass in which the network weights are updated via methods such as a gradient descent method is termed Backpropagation Through Time (BPTT). The deduction of backpropagation rules, i.e., equations for the backpropagation of the gradients from the difference between the predicted and actual outputs, depends on the loss function and activation functions.

In the following, we will derive the general backpropagation equations. In this deduction, we will use the mean square root error for loss, tanh for ${\sigma }_1$${\sigma }_1$sigma_(1)\sigma_{1}${\sigma }_1$, and Sigmoid for ${\sigma }_2$${\sigma }_2$sigma_(2)\sigma_{2}${\sigma }_2$ to show more detailed equations that can be used for implementations. Accordingly, the three functions have the following derivatives, which are needed in the backpropagation.

The gradient of ${\overline{V}}_{K\times N}$${\overline{V}}_{K\times N}$bar(V)_(K xx N)\bar{V}_{K \times N}${\overline{V}}_{K\times N}$ and ${\overrightarrow{c}}_{K\times 1}$${\overrightarrow{c}}_{K\times 1}$vec(c)_(K xx1)\vec{c}_{K \times 1}${\overrightarrow{c}}_{K\times 1}$ can be simply calculated as follows:

The calculation of the gradients for $\overline{W},\overline{U}$$\overline{W},\overline{U}$bar(W), bar(U)\bar{W}, \bar{U}$\overline{W},\overline{U}$, and $\overrightarrow{b}$$\overrightarrow{b}$vec(b)\vec{b}$\overrightarrow{b}$ is more complicated. As illustrated in the unfolded RNN, the gradient at a time step $t$$t$tt$t$ comes from two parts: the loss caused by ${\widetilde{\stackrel{~}{y}}}^t-{\overrightarrow{y}}^t$${\widetilde{\stackrel{~}{y}}}^t-{\overrightarrow{y}}^t$tilde(tilde(y))^(t)- vec(y)^(t)\tilde{\tilde{y}}^{t}-\vec{y}^{t}${\widetilde{\stackrel{~}{y}}}^t-{\overrightarrow{y}}^t$ at the current step and that from the next time step via the hidden layer connection ( ${\overrightarrow{h}}^t$${\overrightarrow{h}}^t$vec(h)^(t)\vec{h}^{t}${\overrightarrow{h}}^t$ and ${\overrightarrow{h}}^{t+1}$${\overrightarrow{h}}^{t+1}$vec(h)^(t+1)\vec{h}^{t+1}${\overrightarrow{h}}^{t+1}$ ). Thus, the gradient calculation will need to start from the last time step $\tau$$\tau$tau\tau$\tau$ and then move backward (time) step by step.
$\overline{W},\overline{U}$$\overline{W},\overline{U}$bar(W), bar(U)\bar{W}, \bar{U}$\overline{W},\overline{U}$, and $\overrightarrow{b}$$\overrightarrow{b}$vec(b)\vec{b}$\overrightarrow{b}$ are contained in the function for ${\overrightarrow{h}}^t$${\overrightarrow{h}}^t$vec(h)^(t)\vec{h}^{t}${\overrightarrow{h}}^t$. Thus, the calculation of the gradients for these parameters will also involve $\frac{\partial \ell }{\partial {\overrightarrow{h}}^t}$$\frac{\partial \ell }{\partial {\overrightarrow{h}}^t}$(delℓ)/(del vec(h)^(t))\frac{\partial \ell}{\partial \vec{h}^{t}}$\frac{\partial \ell }{\partial {\overrightarrow{h}}^t}$ when applying the chain rule. For example, for $\overline{W}$$\overline{W}$bar(W)\bar{W}$\overline{W}$, we have $\frac{\partial \ell }{\partial \overline{W}}{}_{N\times N}=\frac{\partial \ell }{\partial \overrightarrow{h}}{}_{N\times 1}\cdot \frac{\partial \overrightarrow{h}}{\partial \overline{W}}{}_{1\times N}$$\frac{\partial \ell }{\partial \overline{W}}{}_{N\times N}=\frac{\partial \ell }{\partial \overrightarrow{h}}{}_{N\times 1}\cdot \frac{\partial \overrightarrow{h}}{\partial \overline{W}}{}_{1\times N}$(delℓ)/(del( bar(W)))_(N xx N)=(delℓ)/(del( vec(h)))_(N xx1)*(del( vec(h)))/(del( bar(W)))_(1xx N)\frac{\partial \ell}{\partial \bar{W}}{ }_{N \times N}=\frac{\partial \ell}{\partial \vec{h}}{ }_{N \times 1} \cdot \frac{\partial \vec{h}}{\partial \bar{W}}{ }_{1 \times N}$\frac{\partial \ell }{\partial \overline{W}}{}_{N\times N}=\frac{\partial \ell }{\partial \overrightarrow{h}}{}_{N\times 1}\cdot \frac{\partial \overrightarrow{h}}{\partial \overline{W}}{}_{1\times N}$. To simplify the deduction, we can define the following intermediate variable:

Then, we can expand ${\overrightarrow{\delta }}^t$${\overrightarrow{\delta }}^t$vec(delta)^(t)\vec{\delta}^{t}${\overrightarrow{\delta }}^t$ to obtain the recursive relationship between steps.

where the function $\mathrm{diag}(\overrightarrow{h})$$\mathrm{diag}(\overrightarrow{h})$diag( vec(h))\operatorname{diag}(\vec{h})$\mathrm{diag}(\overrightarrow{h})$ converts a vector $\overrightarrow{h}$$\overrightarrow{h}$vec(h)\vec{h}$\overrightarrow{h}$ with a shape of $(N$$(N$(N(N$(N$,$)or(N,1)$$)or(N,1)$)or(N,1)) or (N, 1)$)or(N,1)$ to a 2D array with a shape of $(N,N)$$(N,N)$(N,N)(N, N)$(N,N)$ whose diagonal is $\overrightarrow{h}$$\overrightarrow{h}$vec(h)\vec{h}$\overrightarrow{h}$.
${\overrightarrow{\delta }}^{t+1}$${\overrightarrow{\delta }}^{t+1}$vec(delta)^(t+1)\vec{\delta}^{t+1}${\overrightarrow{\delta }}^{t+1}$ has a special case when $t=\tau$$t=\tau$t=taut=\tau$t=\tau$. At this last step, there is no gradient coming from the next step. Thus, it can be calculated directly.

With ${\overrightarrow{\delta }}^t$${\overrightarrow{\delta }}^t$vec(delta)^(t)\vec{\delta}^{t}${\overrightarrow{\delta }}^t$, the gradients of $\overline{W},\overline{U}$$\overline{W},\overline{U}$bar(W), bar(U)\bar{W}, \bar{U}$\overline{W},\overline{U}$, and $\overrightarrow{b}$$\overrightarrow{b}$vec(b)\vec{b}$\overrightarrow{b}$ can be calculated as

When implementing the above backward pass, we can start from the last time step. First, we calculate $\frac{\partial \ell }{\partial \overrightarrow{c}},\frac{\partial \ell }{\partial \overline{V}}$$\frac{\partial \ell }{\partial \overrightarrow{c}},\frac{\partial \ell }{\partial \overline{V}}$(delℓ)/(del( vec(c))),(delℓ)/(del( bar(V)))\frac{\partial \ell}{\partial \vec{c}}, \frac{\partial \ell}{\partial \bar{V}}$\frac{\partial \ell }{\partial \overrightarrow{c}},\frac{\partial \ell }{\partial \overline{V}}$, and ${\overrightarrow{\delta }}^t$${\overrightarrow{\delta }}^t$vec(delta)^(t)\vec{\delta}^{t}${\overrightarrow{\delta }}^t$. Then, we can compute $\frac{\partial \ell }{\partial \overline{W}},\frac{\partial \ell }{\partial U}$$\frac{\partial \ell }{\partial \overline{W}},\frac{\partial \ell }{\partial U}$(delℓ)/(del( bar(W))),(delℓ)/(del U)\frac{\partial \ell}{\partial \bar{W}}, \frac{\partial \ell}{\partial U}$\frac{\partial \ell }{\partial \overline{W}},\frac{\partial \ell }{\partial U}$, and $\frac{\partial \ell }{\partial \overrightarrow{b}}$$\frac{\partial \ell }{\partial \overrightarrow{b}}$(delℓ)/(del( vec(b)))\frac{\partial \ell}{\partial \vec{b}}$\frac{\partial \ell }{\partial \overrightarrow{b}}$.

All the model parameters, i.e., $\overrightarrow{c}\overline{V},\overline{W},\overline{U}$$\overrightarrow{c}\overline{V},\overline{W},\overline{U}$vec(c) bar(V), bar(W), bar(U)\vec{c} \bar{V}, \bar{W}, \bar{U}$\overrightarrow{c}\overline{V},\overline{W},\overline{U}$, and $\overrightarrow{b}$$\overrightarrow{b}$vec(b)\vec{b}$\overrightarrow{b}$, which can be represented using a general parameter array, i.e., $\theta$$\theta$theta\theta$\theta$, can be updated using the gradient descent:

where $\alpha$$\alpha$alpha\alpha$\alpha$ is the learning rate.